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12 August 1981 


MEMORANDUM FOR: 
Director, Intelligence Community Staff 


VIA: re 
Director, Office of Program and Performance Assessment 


FROM: eee 
eputy Director, Office of Program and Performance 


Assessment 


SUBJECT: Proposal for a Study of Collection Evaluation Systems 


1. Action Requested: That you approve the study described in the 
attached Terms of Reference. 


2. Background: In reviewing recent studies being conducted by OPPA 
on COMINT and HUMINT capabilities in the Third World, I am reminded that 
there are a number of mechanisms in the Community for evaluating how well 
the collection systems satisfy the requirements of intelligence producers. 
For example, the Publication Source Survey (PSS) at NFAC has proved useful 
in stimates of different collectors’ 
contributions to finished intelligence from NFAC, and has 
used statistics from DIA's HUMINT Information Management System (HIMS) to 
analyze Defense attache performance. Based on these experiences, I think 
a survey study describing PSS, the evaluation portions of HIMS, and as 
many other collection evaluation systems as we can identify in the 
Community would serve a number of purposes. Such a study would: 


e allow us to compare the evaluation system(s) used by different 
collection disciplines and suggest ways to improve collection 
evaluation by expanding or modifying existing mechanisms or by 
adding new ones, 


e provide a reference document that we in OPPA could use to ensure 
future, cross-discipline collection evaluations that we might 
perform take advantage of data already available in existing 
systems, and 
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e be a source of ideas for ways to develop and focus staff efforts 
on production evaluation so that whatever we do in the 
production arena links well with existing collection evaluation 
systems. This linkage is essential for better end-to-end 
assessments of intelligence performance. 


3. Recommendation: That you approve the study described in the 


attached Terms of Reference. This memo has been coordinated with the 
Collection Offices. 
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